Character recognition method and apparatus

ABSTRACT

A character recognition method is provided, in which the method includes the steps of: picking up images in a motion picture by changing brightness for each image; extracting a character area from each image; recognizing at least one character in the character area by performing a matching process between the character area and dictionary images for each image, and determining a character recognition result for each image; and selecting a final character recognition result of which the matching degree is the highest from the character recognition results of the images. Accordingly, even when a character is too dark to recognize due to shadows in the daytime or a character is too bright to recognize due to irradiation in the nighttime, the character can be recognized by using images having differing brightness.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to a character recognition methodand apparatus. More particularly, the present invention relates to acharacter recognition method and apparatus for extracting andrecognizing characters in a motion picture.

[0003] 2. Description of the Related Art

[0004] With recent improvements in image processing technology, it hasbecome possible to extract and recognize characters in a motion picture.In addition, a technology has been developed for extracting a characterarea from a recognition target object that passes through a camera rangeand recognizing characters in the character area, in which image pickupdevices and image processing apparatuses are set up in various locationsindoors and/or outdoors.

[0005] For example, the character recognition apparatus can be used fora system for recognizing numbers of cars that enter a parking area tomanage the cars.

[0006]FIG. 1 shows a block diagram of an example of a conventionalcharacter recognition apparatus. Each input image from an image pickupapparatus 10 is captured in a processor 20 by an image capturing part 21every {fraction (1/30)} second. For the captured image, a target objectdetection part 22 checks whether a recognition target object, a car, forexample, appears. An image from which the recognition target object isdetected is passed to a character area extraction processing part 23.The character area extraction processing part 23 extracts a characterarea (for example, an area of a number plate) from the recognitiontarget object. The character recognition processing part 24 performs amatching process between the extracted character area and dictionaryimages, so that recognition result is output to the outside, whereineach character is compared with dictionary images to determine if thereis a match in the matching process, for example.

[0007] Usually, focus and shutter speed of an infrared camera in theimage pickup apparatus 10 are adjusted and fixed when the image pickupapparatus 10 is set up. After that, an iris control part 25 in theprocessor 20 controls, via an image pickup apparatus control part 26, aniris of the infrared camera 11 such that brightness of the characterarea or brightness of the whole image becomes proper for performingrecognition.

[0008] However, depending on circumstances determined by location andsetting conditions of the image pickup apparatus 10 and by a timeperiod, iris control cannot be performed properly, so that characters inthe image cannot be recognized accurately. For example, the iris cannotbe controlled to a proper value instantaneously when shadows ofsurrounding objects during the daytime, irradiation from surroundingobjects during the night, obstacles and the like appear.

[0009]FIG. 2 shows changes of an image of a character area due toenvironmental change, in which (A) corresponds to a normal condition,(B) corresponds to a condition in which shadows of surrounding objectsoccur during the daytime, (C) corresponds to a condition in whichirradiation from surrounding objects occurs during the night, and (D)corresponds to a condition in which the character area is hidden by anobstacle. In the conditions shown in (B), (C), and (D), the character inthe character area cannot be recognized properly.

SUMMARY OF THE INVENTION

[0010] An object of the present invention is to provide a characterrecognition method and apparatus for picking up an image of a characterarea properly in various circumstances and for improving performance ofcharacter recognition.

[0011] The object of the present invention can be achieved by acharacter recognition method used for extracting a character area from arecognition target object in a motion picture and recognizing acharacter in the character area, the character recognition methodincluding the steps of:

[0012] picking up images in a motion picture by changing brightness foreach image;

[0013] extracting a character area from each image;

[0014] recognizing at least a character in the character area byperforming a matching process between the character area and dictionaryimages for each image, and determining a character recognition resultfor each image; and

[0015] selecting a final character recognition result of which thematching degree is the highest from character recognition results of theimages.

[0016] According to the above-mentioned invention, even when a characteris too dark to recognize due to shadows of surrounding objects in thedaytime, the character can be recognized by using images havingdiffering brightness. In addition, even when a character is too brightto recognize due to irradiation from surroundings in the nighttime, thecharacter can be recognized by using images having differing brightness.Thus, character recognition performance improves.

BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Other objects, features and advantages of the present inventionwill become more apparent from the following detailed description whenread in conjunction with the accompanying drawings, in which:

[0018]FIG. 1 shows a block diagram of an example of a conventionalcharacter recognition apparatus;

[0019]FIG. 2 shows changes of an image of a character area due toenvironmental change;

[0020]FIG. 3 shows a block diagram of the first embodiment of acharacter recognition apparatus of the present invention;

[0021]FIG. 4 shows a flowchart showing processes performed by aprocessor according to the present invention;

[0022]FIG. 5 shows differences of character status between a normalimage and an additional image in the daytime;

[0023]FIG. 6 shows differences of character status between a normalimage and an additional image in the nighttime;

[0024]FIG. 7 shows a block diagram of the second embodiment of thecharacter recognition apparatus according to the present invention;

[0025]FIG. 8 shows a block diagram of the third embodiment of thecharacter recognition apparatus according to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026]FIG. 3 shows a block diagram of the first embodiment of acharacter recognition apparatus of the present invention. In the figure,an image pickup apparatus 30 includes an infrared camera 31 that canpick up images day and night, and an illuminating LED 32. The infraredcamera 31 has a function to use different shutter speeds for each imageof an odd field and an even field. The illuminating LED 32 is effectiveonly for images of the odd field.

[0027] An odd field capturing part 41A and an even field capturing part41B in a processor 40 capture the odd field and the even field,respectively almost at the same time, from an input image from the imagepickup apparatus 30 every {fraction (1/30)} second. After that, the oddfield image is processed as a normal image, and the even field image isprocessed as an additional image.

[0028] An image captured by the odd field capturing part 41A is suppliedto a target object detection part 42, in which it is determined whethera recognition target object such as a car exists. An odd field image inwhich the recognition target object is detected is passed to a characterarea extraction processing part 43, in which a character area (a numberplate, for example) is extracted from the recognition target object.Also, as for the even field image, the recognition target object isregarded to be detected in a position almost the same as that where therecognition target object is detected in the odd field image, and theeven field image is passed to the character area extraction processingpart 43, and a character area is extracted from the recognition targetobject.

[0029] For example, in the character area extraction processing part 43,an area where the number of edges of perpendicular directions exceeds athreshold is extracted as the character area, wherein an edge is a pointwhere brightness rapidly changes in a monochrome image. This extractionmethod utilizes a property that the number of edges of perpendiculardirections in a character area is greater than that in other areas.

[0030] Each character area in the extracted odd field image and evenfield image is compared with dictionary images in the dictionary imagestoring part 48 by the character recognition processing part 44 as amatching process, and a character matching with a dictionary image ispassed to a result selection part 45. The result selection part 45selects a character recognition result that matches the dictionary imagemore accurately between a character recognition result of the odd fieldand a character recognition result of the even field. In the following,accuracy of matching will be called “matching degree”.

[0031] Normally, focus and shutter speed of the infrared camera 31 inthe image pickup apparatus 30 are adjusted and fixed when the imagepickup apparatus 30 is set up. The iris control part 46 in the processor40 controls the iris of the infrared camera 31 via the image pickupcontrol part 47 such that brightness of the character area to beextracted from a normal image of the odd field (or whole image of thenormal image) becomes a proper value.

[0032] In addition, on the basis of an iris setting value obtained here,the illuminating LED 32 and shutter speed for an additional image arecontrolled. When the iris setting value is no less than a predeterminedthreshold, the apparatus determines that it is daytime, the illuminatingLED 32 is turned off, and the shutter speed for the additional image isset to be ½ of that for a normal image or less. When the iris settingvalue is smaller than the threshold, the apparatus determines that it isnight, so that the illuminating LED 32 is turned on and the shutterspeed for the additional image is set to be the same as that for thenormal image.

[0033]FIG. 4 shows a flowchart showing processes performed by theprocessor 40. As shown in the figure, a normal image (odd field image)and an additional image (even field image) are captured every {fraction(1/30)} second in step 10, and a target object detection process isperformed by using the normal image in step 12.

[0034] It is determined whether the target object is detected in step14. When the target object is not detected, the process goes to step 10.When the target object is detected, a process for extracting a characterarea is performed for the normal image in step 16. Then, it isdetermined whether the character area is extracted or not in step 18.When the character area is not detected, the process goes back to step10.

[0035] When the character area is extracted from the normal image, thecharacter recognition process is performed in step 20 for the characterarea by a dictionary matching process, so that recognition result of thenormal image is obtained. Concurrently with this character recognitionprocess, a position of the character area in the additional image isdetermined in step 22, and character recognition process is performedfor the character area of the additional image in step 24, so thatrecognition result of the additional image is obtained. Since the normalimage and the additional image are images in a same frame, it is assumedthat the position of the character area in the additional image isalmost the same as that in the normal image.

[0036] As for the recognition results obtained in steps 20 and 24,matching degrees are compared for each character in step 26. And, animage having a higher matching degree is determined as a finalrecognition result and output in step 28.

[0037] In addition, concurrently with steps 20, 22 and 24, the iris ofthe infrared camera 31 is adjusted such that brightness of the characterarea extracted from the normal image becomes a proper value in step 30.Steps 30-36 may be performed after the steps 20-28 are performed.

[0038] In step 32, it is determined whether the iris setting value is noless than a threshold. When the iris setting value is no less than thethreshold, it is determined that it is daytime, so that the illuminatingLED 32 is turned off, and shutter speed for the additional image is setto be ½ of that of the normal image (or slower). For example, if shutterspeed for the normal image is {fraction (1/1000)} (sec), shutter speedfor the additional image is set to be {fraction (1/500)} (sec).

[0039] When the iris setting value is smaller than the threshold, it isdetermined that it is nighttime, so that the illuminating LED 32 isturned on in step 36 only for the normal image, and shutter speed forthe additional image is set to be the same as that of the normal image.

[0040] In the above-mentioned control, by setting shutter speed for theadditional image to be ½ of that for the normal image in daytime, theadditional image becomes brighter than the normal image. In thenighttime, by turning on the illuminating LED 32 (only for the odd fieldimage), the additional image becomes darker than the normal image.

[0041] When the image pickup apparatus turns on the illuminating LED 32for both fields, the same control can be realized by setting the shutterspeed for the additional image to be two times (or more) as that of thenormal image. Thus, recognition performance in the nighttime can beimproved even if the image pickup apparatus does not have the functionto adjust shutter speed for each field.

[0042]FIG. 5 shows differences of character status between the normalimage and the additional image in the daytime. In the daytime, since theadditional image is brighter than the normal image, even if a characterin the normal image is too dark to recognize properly due to shadows ofsurrounding objects, the character can be recognized properly in theadditional image as shown in FIG. 5.

[0043]FIG. 6 shows differences of character status between the normalimage and the additional image in the nighttime. In the nighttime, sincethe additional image is darker than the normal image, even when acharacter in the normal image is too bright to recognize properly due tolight irradiation from surroundings, the character can be recognizedproperly in the additional image.

[0044] In the above-mentioned method of the present invention, even whenbrightness near the character area is not proper for recognition in thenormal image, brightness near the character area becomes proper forrecognition in the additional image. Thus, stable character recognitionperformance can be kept, so that recognition performance improves.

[0045] Although the normal image and the additional image are capturedas the odd field and the even field in the above-mentioned embodiment,the normal image and the additional image may be captured as differentframes.

[0046]FIG. 7 shows a block diagram of a second embodiment of thecharacter recognition apparatus according to the present invention. Inthe figure, the same numerals are assigned to the same parts as those inFIG. 3. In FIG. 7, the image pickup apparatus 30 includes an infraredcamera 31 which can be used for day and night and the illuminating LED32. In this embodiment, the illuminating LED 32 is effective for bothodd and even fields.

[0047] The image taken by the infrared camera 31 is provided to an imagecapturing part 51 and a delay part 52 in the processor 50. The imagecapturing part 51 captures each input image from the image pickupapparatus 30 every {fraction (1/30)} second for each field or for eachframe. The image delayed by a delay part 52 for a predetermined time(from several frames to several tens of frames, for example) is providedto an image capturing part 53. The image capturing part 53 captures thedelayed input image every {fraction (1/30)} second for each frame. Afterthat, the image captured by the image capturing part 51 is processed asa present image, and the image captured by the image capturing part 53is processed as a delayed image.

[0048] Each image captured by the image capturing parts 51 and 53 issupplied to a target object detection part 54. The target objectdetection part 54 determines whether a recognition target object existsin each of the present image and the delayed image. The present imageand the delayed image in which the recognition target object is detectedare passed to the character area extraction processing part 43, so thata character area (a number plate, for example), which is the recognitiontarget, is extracted from the recognition target object.

[0049] A matching process is performed for each character area for thepresent image and the delayed image, in which each character area iscompared with the dictionary images in the dictionary image storing part48 by the character recognition processing part 44. Then, charactersmatched with the dictionary image are passed to the result selectionpart 45. The result selection part 45 selects the character recognitionresult that has a higher matching degree between the characterrecognition results of the present image and the delayed image, andoutputs the character recognition result.

[0050] Normally, focus and shutter speed of the infrared camera 31 inthe image pickup apparatus 30 are adjusted and fixed when the imagepickup apparatus 30 is set up. The iris control part 46 in the processor40 controls the iris of the infrared camera 31 via the image pickupapparatus control part 47 such that brightness of the character areaextracted from the present image becomes a proper value. In addition,the iris control part 46 determines that it is daytime when the irissetting value is no less than a predetermined threshold and turns offthe illuminating LED 32. When the iris setting value is smaller than thethreshold, the iris control part 46 determines that it is nighttime andturns on the illuminating LED 32.

[0051] Recognition accuracy improves by selecting a recognition resulton the basis of the matching degree between recognition results ofimages having time difference of several frames. In this case, even whena character cannot be recognized in the present image due to anobstacle, the character can be recognized by using the delayed image inwhich the position of the character is different from that in thepresent image. Accordingly, the possibility for recognizing a characterarea increases even when an obstacle hides the character areaaccidentally, so that character recognition performance furtherimproves.

[0052]FIG. 8 shows a block diagram of the third embodiment of thecharacter recognition apparatus according to the present invention. Inthe figure, the same numerals are assigned to the same parts as those inFIG. 3 and FIG. 7. In FIG. 8, the image pickup apparatus 30 includes aninfrared camera 31 that can be used day and night and the illuminatingLED 32. In this embodiment, the illuminating LED 32 is effective forboth odd and even fields.

[0053] Input images taken by the infrared camera 31 are captured in abuffer 61 successively by each frame. The image captured by the buffer61 is supplied to the target object detection part 64. The target objectdetection part 64 checks whether a recognition target object exists inan upper image, wherein the upper image is one of three images intowhich the input image is divided vertically, the rest of the threeimages being a central image and a lower image. When a recognitiontarget object is detected, the target object recognition part 64 passesthe image to the character area extraction processing part 43, and thebuffer for capturing an input image is changed from the buffer 61 to thebuffer 62.

[0054] The character area extraction processing part 43 extracts ancharacter area from the recognition target object, the extractedcharacter area is compared with dictionary images in the dictionaryimage storing part 48 by the character recognition processing part 44.Then, characters that match with dictionary images are passed to theresult selection part 45.

[0055] The image captured in the buffer 62 is supplied to the targetobject detection part 64. The target object detection part 64 determineswhether a recognition target object exists in the central image. When arecognition target object is detected, the target object recognitionpart 64 passes the image to the character area extraction processingpart 43, and the buffer for capturing the input image is changed fromthe buffer 62 to the buffer 63.

[0056] The character area extraction processing part 43 extracts acharacter area from the recognition target object, the extractedcharacter object is compared with dictionary image by the charactersrecognition processing part 44. Then, characters that match withdictionary image are passed to the result selection part 45.

[0057] The image captured in the buffer 63 is supplied to the targetobject detection part 64. The target object detection part 64 checkswhether a recognition target object exists in the lower image. When arecognition target object is detected, the target object recognitionpart 64 passes the image to the character area extraction processingpart 43, and the buffer for capturing the input image is changed fromthe buffer 63 to the buffer 61.

[0058] The character area extraction processing part 43 extracts acharacter area from the recognition target object, the extractedcharacter object is compared with dictionary image by the characterrecognition processing part 44. Then, characters that match with thedictionary image are passed to the result selection part 45.

[0059] The result selection part 45 selects the character recognitionresult that has the highest matching degree, that is, that matches thedictionary image most accurately among character recognition results ofthe upper image, the central image and the lower image, and outputs theselected result to the outside, and notifies the image pickup apparatuscontrol part 65 of the result. After that, character recognition foreach of the upper image, central image and the lower image is repeated.

[0060] Normally, focus and shutter speed of the infrared camera 31 inthe image pickup apparatus 30 are adjusted and fixed when the imagepickup apparatus 30 is set up. The iris control part 45 in the processor40 controls the iris of the infrared camera 31 via the image pickupapparatus control part 65 such that brightness of each character areaextracted from the upper image, the central image and the lower imagebecomes a proper value. In addition, the iris control part 46 determinesthat it is daytime when the iris setting value is no less than apredetermined threshold and the iris control part 46 turns off theilluminating LED 32. When the iris setting value is smaller than athreshold, the iris control part 46 determines that it is nighttime sothat the illuminating LED 32 is turned on.

[0061] In addition, the image pickup control part 65 performs focuschange control to adjust the focus of the infrared camera 31 to fartherby a predetermined ratio if the selected image is the upper image, andto adjust the focus to nearer by a predetermined ratio if the selectedimage is the lower image.

[0062] Generally, the upper image includes farther objects than those inthe central image, and the lower image includes nearer objects thanthose in the central image. Therefore, by performing the focus changecontrol, the focus of the infrared camera 31 can be automaticallycontrolled such that the image having the highest degree of matchingbecomes the central image. Accordingly, even when the focus is notadjusted properly when setting up the image pickup apparatus 30, thefocus can be adjusted automatically.

[0063] The present invention is applied not only to the system forrecognizing numbers of cars entering a parking and managing the cars,but also to various systems such as a system for recognizing numbers ofcars entering a toll gate of a highway, and a system for recognizingnumbers and destinations of commodities conveyed by a belt conveyor.

[0064] As mentioned above, according to the present invention, even whena character is too dark to recognize due to shadows of surroundingobjects in the daytime, the character can be recognized by using imageshaving differing brightness. In addition, even when a character is toobright to recognize due to irradiation from surroundings in thenighttime, the character can be recognized by using images havingdiffering brightness.

[0065] In addition, according to the present invention, since imageshaving time difference with each other are used for characterrecognition and the final character recognition result of which thematching degree is the highest is selected from character recognitionresults of the images, the possibility for recognizing a character areaincreases even when an obstacle hides the character area accidentally,so that character recognition performance further improves.

[0066] Further, according to the present invention, an image in a motionpicture is divided into a plurality of regions including an upper regionand an lower region for character recognition of each region, a finalcharacter recognition result of which the matching degree is the highestis selected from character recognition results of the regions, and focusof an image pickup apparatus is adjusted to focus on farther objectswhen the final character recognition is for the upper region, and thefocus is adjusted to focus on nearer objects when the final characterrecognition is for the lower region. Therefore, even when focus of theimage pickup apparatus is not adjusted properly at the time of set up,the focus can be automatically adjusted.

[0067] The present invention is not limited to the specificallydisclosed embodiments, and variations and modifications may be madewithout departing from the scope of the invention.

What is claimed is:
 1. A character recognition method used forextracting a character area from a recognition target object in a motionpicture and recognizing a character in said character area, saidcharacter recognition method comprising the steps of: picking up imagesin the motion picture by changing brightness for each of the images;extracting at least one character area from each image; recognizing atleast one character in said character area by performing a matchingprocess between said character area and dictionary images for eachimage, and determining a character recognition result for each image;and selecting a final character recognition result of which a matchingdegree is the highest from the character recognition results of saidimages.
 2. A character recognition method used for extracting acharacter area from a recognition target object in a motion picture andrecognizing a character in said character area, said characterrecognition method comprising the steps of: extracting images havingtime difference with each other from the motion picture; extracting saidcharacter area from each image; recognizing at least one character insaid character area by performing a matching process between saidcharacter area and dictionary images for each image, and determining acharacter recognition result for each image; and selecting a finalcharacter recognition result of which a matching degree is the highestfrom the character recognition results of said images.
 3. A characterrecognition method used for extracting a character area from arecognition target object in a motion picture and recognizing acharacter in said character area, said character recognition methodcomprising the steps of: dividing an image in the motion picture into aplurality of regions including an upper region and a lower region;extracting said character area from each region; recognizing at leastone character in said character area by performing a matching processbetween said character area and dictionary images for each region, anddetermining a character recognition result for each region; selecting afinal character recognition result of which a matching degree is thehighest from the character recognition results of said regions; andadjusting a focus of an image pickup apparatus to farther objects whensaid final character recognition is for said upper region, and adjustingsaid focus to nearer objects when said final character recognition isfor said lower region.
 4. A character recognition apparatus used forextracting a character area from a recognition target object in a motionpicture and recognizing a character in said character area, saidcharacter recognition apparatus comprising: a part for picking up imagesin the motion picture by changing brightness for each image; a part forextracting said character area from each image; a part for recognizingat least one character in said character area by performing a matchingprocess between said character area and dictionary images for eachimage, and determining a character recognition result for each image;and a part for selecting a final character recognition result of which amatching degree is the highest from the character recognition results ofsaid images.
 5. The character recognition apparatus as claimed in claim4, said part for picking up images including a part for changing ashutter speed to change brightness for each image.
 6. The characterrecognition apparatus as claimed in claim 4, said part for picking upimages including a part for changing illumination to change brightnessfor each image.
 7. A character recognition apparatus used for extractinga character area from a recognition target object in a motion pictureand recognizing a character in said character area, said characterrecognition apparatus comprising: a part for extracting images havingtime difference with each other from the motion picture; a part forextracting said character area from each image; a part for recognizingat least one character in said character area by performing a matchingprocess between said character area and dictionary images for eachimage, and determining a character recognition result for each image;and a part for selecting a final character recognition result of which amatching degree is the highest from the character recognition results ofsaid images.
 8. A character recognition apparatus used for extracting acharacter area from a recognition target object in a motion picture andrecognizing a character in said character area, said characterrecognition apparatus comprising: a part for dividing an image in themotion picture into a plurality of regions including an upper region anda lower region; a part for extracting said character area from eachregion; a part for recognizing at least one character in said characterarea by performing a matching process between said character area anddictionary images for each region, and determining a characterrecognition result for each region; a part for selecting a finalcharacter recognition result of which a matching degree is the highestfrom the character recognition results of said regions; and a part foradjusting a focus of an image pickup apparatus to farther objects whensaid final character recognition is for said upper region, and adjustingsaid focus to nearer objects when said final character recognition isfor said lower region.